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Introduction 


¢ The importance of a realistic covariance has grown in recent years 
due to its inclusion in conjunction analysis. 


¢ The GSFC FDF has been responsible for providing the definitive orbit 
solutions for the Aqua and Aura missions for over a decade. 


— The orbit solutions utilizes Doppler data from ground stations and coherent 
range and Doppler from the NASA Space Network. 


¢ The NASA Aqua and Aura missions have requested the covariance 
data be delivered, so that it can be used by the NASA conjunction 
assessment (CA) team. 


¢ This prompted an analysis of the covariance coming out of the 
operational extended Kalman filter (EFK). 


¢ Definitive and predictive accuracy of the orbit solutions remains an 
important priority for supporting science operations. 
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The Chi-squared Statistic 


¢ The Chi-squared value is calculated in an attempt to characterize 
how well the covariance conforms to the actual error of a state. 


¢ There are two inputs involved in the calculation 


— Error inthe state 
¢ Calculated by comparing the predicted state to the definitive state 
° €= [Ep & Ec] 


— Covariance values 


¢ Position covariance taken directly from Orbit Determination Toolkit (ODTK) EKF 
values and converted to the RIC frame 


“Cc 
¢ The chi-squared value is calculated as follows 


7 X3 aof =eC te! 
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The Chi-squared Statistic (cont’d) 


¢ Inan ideal case, the error in the predicted state at any given point 
will match the error represented in the covariance matrix. 


¢ For the three degree of freedom case of a satellite’s orbit, the chi- 
squared should have an average value of 3 and a standard 
deviation of 2.333. 


¢ Average values greater than 3 indicate the covariance is not 
capturing the actual error in the propagation, while values less 
than 3 indicate that the covariance tends to be larger than the 
actual error 


¢ Detailed treatment of the practical application of this statistic is 
available in the technical memorandum “Covariance Realism 
Evaluation Approaches” published by M.D. Hejduk 
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Baseline Scenario 
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Tuning Process 


¢ The tuning process focused on adjusting the EKF parameters in 
order to improve both the realism of the covariance, as well as the 
predictive accuracy of the solution. 


¢ Methods used include 
¢ Adjusting the white noise sigma values associated with tracking data measurements 


* Implementing updated drag models 
¢ Injecting process noise into the scenario to inflate the covariance 
¢ The tuning took place in two phases 
— The first phase focused on improving the predictive accuracy of the solution 


— The second focused on injecting process noise to appropriately size the 
covariance while retaining improvements in predictive accuracy 
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Tuning Predictive Accuracy 


¢ The white noise sigma values associated with tracking data 
measurements were drastically reduced 


— The white noise values associated with the ground station Doppler 
measurements were dropped from 5 cm/sec to an average value of 0.5 cm/sec. 
¢ Each station was tuned individually 


— Shrinking the error associated with the SN measurements proved more 
complicated due to the fact that ODTK uses the actual covariance of the TDRS 
spacecraft as an input to its noise calculations 

¢ ODTK allows the noise to be reduced based on the final collection point of the SN 


measurement, but the majority of the noise in the system existed in the state 
uncertainty of the TDRS 


¢ Reducing the TDRS uncertainty involved tuning the ground based measurements 
used to determine the TDRS orbit state. 
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¢ The final result was a significant reduction in excess measurement 
noise in the filter 
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¢ Implementing updated drag models 


— A box and wing area plugin was implemented for both Aqua and Aura 
spacecraft assuming constant sun pointing for the solar panels 


— The result was actually a degradation in predictive accuracy. Further 
investigation is needed into why the modeling did not perform as expected 
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Injecting Process Noise 


Measurement tuning resulted in chi-squared average values over 8 
for both spacecraft. 


To counteract this, measurement noise values were increased but 
doing this also appreciably degraded predictive accuracy . 


The focus turned to injecting process noise during propagation to 
effectively inflate the covariance. 


Further simplifying the equation for the chi-squared statistic can 
supply insight into the behavior in the individual components 


— Assume that the off-diagonal term of the covariance are zero and the equation 
can be reduced to 


2 ER 7 EC 
X3 dof — 72 + at 
f oz 
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Injecting Process Noise (cont’d) 


¢ From this equation it can be deduced that to achieve a chi-squared 
value of three, the predictive error and sigma values should have a 
one to one relationship. 


¢ Process noise then should be injected with the goal being that the 
average sigma value matches the average predictive error ina 
certain component. 


¢ To this effect, tuning runs were evaluated on a component basis in 
the radial, in-track and cross-track components 


— If the distribution of the error divided by the sigma (“Error over Sigma”, or EOS) 
was 60 percent or higher, favoring either the predictive error or the sigma value, 
the process noise in that component was adjusted. 


— Analysis runs were completed until all of the components met this threshold for 
both spacecraft. 
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Results 


Process Noise Injected (cm/sec) Prediction Error Chi-Squared 
(meters) Average 


Aqua AT: 0.075, CT: 0.001 8.77 
Aura AT: 0.075, CT: 0.001 202 6.50 
2 Aqua AT: 0.060, CT: 0.001 185 8.77 
Aura AT: 0.075, CT: 0.002 202 6.50 
3 Aqua AT: 0.060, CT: 0.003 191 8.2 
Aura AT: 0.075, CT: 0.005 203 5.56 
4 Aqua AT: 0.075, CT: 0.300 179 6.59 
Aura AT: 0.075, CT: 0.400 204 4.84 
5 Aqua AT: 0.050, CT: 0.300 181 6.63 
Aura AT: 0.060, CT: 0.400 189 5:19 
6 Aqua AT: 0.030, CT: 0.300 193 7.59 
Aura AT: 0.030, CT: 0.400 185 5.79 
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Cumulative Density 


Results (cont’d) 
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Conclusions 


¢ The process noise injection process applied did not produce an 
acceptable level of covariance realism. 


— Upon inspection it was clear that while the average predictive error and sigma 
values were close to a one to one relationship, the standard deviation of the 


predictive error was much higher than the standard deviation of the sigma 
value. 


¢ Multiple metrics should be used to determine the effectiveness of a 
filter 


— The baseline scenario shows a good covariance realism but it is aided by 
measurement noise that is inflated to higher than realistic levels. 


— The runs with tightened measurement noise show better predictive accuracy 


but the chi-squared values show the covariance is not capturing the error in the 
propagated state. 
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Conclusions 


¢ The dynamics of the problem play an important role in determining 
the key filter metrics 


— Both Aqua and Aura predictive error is heavily dominated by the in-track 
direction. This made tuning of the other components difficult due to the small 
nature of the values in those components. 

— \|t also places extra importance of effectively tuning the covariance in the in- 
track component, since it is the dominant source of error. 
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Future Work 


¢ Investigate the impact of further tuning the process noise to 
capture the high standard deviation in the predictive error. 


— Tighten the thresholds on the EOS value so the process noise can be more 
effectively tuned 


¢ Investigate techniques that would allow the covariance to scale 
appropriately with the predictive error standard deviation. 


— Possibly inject some process noise based upon the input to the atmospheric 
model 


¢ Determine how the state of the solar cycle effects the performance 
of the filter in both predictive accuracy and covariance realism 
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Questions? 


Thank you 
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